High dynamic range image/video coding

ABSTRACT

A disclosed configuration includes a system (or a computer implemented method or a non-transitory computer readable medium) for automatically preprocessing higher dynamic range image data into lower dynamic range image data through a data adaptive tuning process. By automatically preprocessing the higher dynamic range image data into the lower dynamic range image data through the data adaptive tuning process, an existing encoding process for encoding the standard dynamic range image data can be applied to the lower dynamic range image data while preserving metadata sufficient to recover image fidelity even in the high dynamic range. In one aspect, the system (or a computer implemented method or a non-transitory computer readable medium) provides for backwards compatibility between high dynamic range video services and existing standard dynamic range services. In one aspect, regrading is applied in a domain that is perceptually more uniform than the domain it is initially presented.

CROSS-REFERENCE TO RELATED APPLICATION

This application is continuation of U.S. patent application Ser. No.15/173,551, filed on Jun. 3, 2016, which claims priority under 35 U.S.C.§ 119(e) from U.S. Provisional Patent Application No. 62/171,385,entitled “Methods and Apparatus for High Dynamic Range Video Coding”filed on Jun. 5, 2015, each of which is incorporated by reference hereinin its entirety.

BACKGROUND

The disclosure generally relates to the field of image and videocompression, and more specifically to high dynamic range coding ofdigital images and videos.

Proliferation of display technology combined with bandwidthconsiderations prompts standardization of a format of image data andencoding/decoding thereof. Image data representing an image can bestored in a standardized format to allow electronic displays withdifferent types or electronic displays from different manufactures toproperly display the image. For example, image data conforming to ITU-RRec. BT.709 standard may represent luminosity up to 100 nits, andinclude values of color components in a red-green-blue (RGB) colorspace. The image data conforming to conventional standard (e.g., ITU-RRec. BT.709) can be encoded, and encoded data can be decoded using astandard codec (e.g., Main10 High Efficiency Video Coding (HEVC)) fortransmission to another device.

Recent interests in the broadcast and related video communicationsindustry have turned to high dynamic range image data that may representimagery at much higher levels of luminosity, of at least 1,000 nits andpotentially up to 10,000 nits, and with a much wider color gamut, givenby the BT.2020 standard.

However, there is a lack of an efficient encoding and decoding processof the high dynamic range image data. Developing a dedicated encodingand decoding process for high dynamic range image data may be costinefficient. In addition, such dedicated encoding and decoding processmay not be compatible with standard dynamic range image data. A simplerescaling (or lowering) of a dynamic range in an attempt to utilize aconventional encoding and decoding process for standard dynamic rangeimage data conforming to the conventional standard (e.g., ITU-R Rec.BT.709) may not be a viable solution. Assuming for an example,converting image data (or video data) capable of representing up toluminance of 1000 nits into 100 nits can be simply performed by dividingall samples by 10. However, such an approach may entail a loss ofartistic intent or artistic details of the image (or the video), therebydegrading a quality of the image presented. How to regrade an image fromone dynamic range to another while maintaining artistic intent ordetails is a challenging process.

Therefore, the high dynamic range image data having a higher dynamicrange compared to the standard dynamic range image data cannot beprocessed in an efficient manner through a conventional encoding anddecoding approach.

SUMMARY

One or more embodiments of a disclosed configuration include a system(or a computer implemented method or a non-transitory computer readablemedium) for automatically preprocessing a higher dynamic range imagedata into a lower dynamic range image data through a data adaptivetuning process. In one example, the higher dynamic range image data mayrepresent luminosity of at least 1000 nits, and even up to 10,000 nits.In one example, the lower dynamic range image data may be compatiblewith a standard dynamic range image data having a dynamic range capableof representing luminosity up to 100 nits; other examples range up to atmost 500 nits. By automatically preprocessing the higher dynamic rangeimage data into the lower dynamic range image data through the dataadaptive tuning process, an existing encoding process for encoding thestandard dynamic range image data can be applied to the lower dynamicrange image data while preserving metadata sufficient to recover imagefidelity even in the high dynamic range. In one aspect, the system (or acomputer implemented method or a non-transitory computer readablemedium) provides for backwards compatibility between high dynamic rangevideo services and existing standard dynamic range services. In oneaspect, regrading is applied in a domain that is perceptually moreuniform than the domain in which it is initially presented.

In one or more embodiments, a method of encoding high-dynamic rangeimage data is disclosed. The method includes converting the high-dynamicrange image data represented in a red-green-blue (RGB) color space to aYUV color space; obtaining a luminance signal from the convertedhigh-dynamic range image data; generating a smoothed luminance signalbased on the luminance signal; generating a base signal based on thesmoothed luminance signal, the smoothed luminance signal indicative of asurplus of the high-dynamic range image data over the base signal;generating a regraded base signal by performing adaptive monotonicnonlinear mapping with data adaptive tuning to the base signal;performing a color transformation on the regraded base signal;downsampling chroma components of the color transformed regraded basesignal; encoding the downsampled signal to generate an encoded basesignal; generating metadata describing the smoothed luminance signal;and generating a single stream including the encoded base signal and themetadata describing the smoothed luminance signal.

In another embodiment, the steps in the method are performed in adifferent order. In one embodiment, the base signal is color transformedby a color conversion and downsampled prior to the adaptive monotonicnonlinear mapping. In one embodiment, the conversion to YUV 4:2:0 formatis performed prior to the adaptive monotonic nonlinear mapping. In oneembodiment, the adaptive monotonic nonlinear mapping is applied after aconversion to YUV 4:2:2 format, and the downconversion to YUV 4:2:0format is performed afterwards, to prepare for encoding.

In another embodiment, if an initial high dynamic range image data ispresented in a format other than the linear light RGB data, the initialhigh dynamic range image data is first converted to linear light RGBdata, and the disclosed method is applied. In another embodiment, if theinitial high dynamic range image data is presented in linear light butin a YUV 4:2:2 format or 4:2:0 format, the format of the initial highdynamic range image data is maintained, and the method is adapted tothat color format.

In one embodiment, generating the smoothed luminance signal comprisesautomatically determining a selected number of binsizes (i.e., sizes ofbins) of a luminosity histogram of the smoothed luminance signalaccording to a luminance distribution characteristic of the luminancesignal, and generating the luminosity histogram of the smoothedluminance signal according to the selected number of binsizes.

In one embodiment, generating the smoothed luminance signal includesthree steps: transforming the luminance signal of the base signal basedon an opto-electrical transfer function (OETF) to obtain an OETFluminance signal; performing a tone mapping on the transformed luminancesignal based on the luminance distribution characteristic of the OETFluminance signal; and transforming the tone mapped OETF luminance signalbased on an electro-optical transfer function (EOTF), theelectro-optical transfer function being inverse to the OETF. In oneembodiment, the tone mapping on the OETF luminance signal is either anOETF or a gamut function from BT.2020 or BT.709. In one embodiment, thetone mapping is an adaptive nonlinear function design based onstatistics (e.g., mean, covariance) of the OETF luminance signal, andgenerated for example using piecewise polynomial approximations of thehistogram of the luminance signal, and quantized into a finite number ofbins. In one embodiment, the smoothed luminance signal can be generatedby a piecewise polynomial function with a finite number of segments, byapproximating each of the steps above.

In one embodiment, a similar data adaptive monotonic nonlinear processmay be applied to the base signal. In some embodiments, one or moresteps applied to the base signal may be different from generating thesmoothed luminance signal.

In one embodiment, the standard codec is compliant with one ofITU/ISO/IEC HEVC/H.265, ITU/ISO/IEC AVC/H.264, ITU/ISO/IEC MPEG-2/H.262,Alliance for Open Media, Google VPX, and Xiph Theora coding standards.

In one or more embodiment, a method of encoding high-dynamic range imagedata is disclosed. The method comprises: converting the high-dynamicrange image data represented in a red-green-blue (RGB) color space to aYUV color space; obtaining a luminance signal from the convertedhigh-dynamic range image data; generating a smoothed luminance signalbased on the luminance signal; generating a base signal based on thesmoothed luminance signal, the smoothed luminance signal indicative of asurplus of the high-dynamic range image data over the base signal;performing a color transformation on the base signal; downsamplingchroma components of the color transformed base signal; generating aregraded base signal by performing adaptive monotonic nonlinear mappingwith data adaptive tuning to the downsampled base signal; encoding theregraded base signal to generate an encoded base signal; generatingmetadata describing the smoothed luminance signal; and generating asingle stream including the encoded base signal and the metadatadescribing the smoothed luminance signal.

In one embodiment, performing the adaptive monotonic nonlinear mappingwith the data adaptive tuning comprises: transforming a luminancecomponent of the downsampled base signal based on an opto-electricaltransfer function (OETF) to obtain an OETF luminance signal; performinga tone mapping on the transformed luminance signal based on adistribution characteristic of the OETF luminance signal; andtransforming the tone mapped base signal based on an electro-opticaltransfer function, the electro-optical transfer function being inverseto the opto-electrical transfer function.

In one or more embodiments, a method of transmitting image data from asource device to a sink device is disclosed. The method includes:converting the image data represented in a red-green-blue (RGB) colorspace to a YUV color space; generating a smoothed luminance signal basedon a luminance component of the converted image data; generating a basesignal based on the smoothed luminance signal, the smoothed luminancesignal indicative of a surplus in a dynamic range of the image data overthe base signal; generating a regraded base signal by performingadaptive monotonic nonlinear mapping with data adaptive tuning based onthe base signal; encoding the regraded base signal to generate a firststream; and transmitting the first stream to the sink device.

In one embodiment, generating the smoothed luminance signal is based ona luminance a distribution characteristic of the luminance component ofthe converted image data. In one embodiment, a selected number ofbinsizes of a luminosity histogram is automatically determined accordingto the luminance distribution characteristic of the luminance component,and generating the luminosity histogram of the smoothed luminance signalaccording to the selected number of binsizes.

In one embodiment, performing the adaptive monotonic nonlinear mappingwith the data adaptive tuning is based on a sequence, GOP, frame, slice,or even block-level of the base signal.

In one embodiment, the smoothed luminance signal is encoded to generatemetadata describing the smoothed luminance signal. The first stream maybe generated by encoding the regraded base signal and adding themetadata to the encoded base signal.

In one embodiment, the smoothed luminance signal is encoded to generatea second stream different from the first stream, and the second streamis transmitted to the sink device.

In one embodiment, metadata including parameters used for generating thesmoothed luminance signal is generated. The first stream may begenerated by encoding the regraded base signal and adding the metadatato the encoded base signal. The parameters may include one of an inputpeak brightness, a target peak brightness, a number of bins, andbinsizes that are adjusted according to the luminance component.

In one embodiment, the image data can represent up to 1000 nits, and thebase signal can represent up to 100 nits. In another embodiment, theimage data can represent up to 4000 nits, and wherein the base signalcan represent up to 1000 nits. In one embodiment, the method allowschanging the peak luminosity from a given value A to a given value B,where A>B or A<B. For example, A can be 4000, and B can be 500; or theother way.

In one embodiment, the base signal is generated by dividing the higherdynamic range image data represented in the RGB color space by thesmoothed luminance signal. The base signal and the regraded base signalmay be represented in the RGB color space. The regraded base signalrepresented in the RGB color space may be converted into the YUV colorspace prior to the encoding to obtain the first stream.

In one embodiment, the base signal is generated by dividing theconverted higher dynamic range image data represented in the YUV colorspace by the smoothed luminance signal. The base signal and the regradedbase signal may be represented in the YUV color space.

One or more embodiments of the disclosed configuration include a system(or a computer implemented method or a non-transitory computer readablemedium) for automatically postprocessing lower dynamic range image datainto high dynamic range image data. By automatically postprocessing thelower dynamic range image data into the higher dynamic range image data,an existing encoding process (e.g., HEVC) for decoding standard dynamicrange image data can be applied to the lower dynamic range image datafor presentation of the high dynamic range image data at a displaydevice.

In one or more embodiments, a method of generating high-dynamic rangeimage data is disclosed. The method includes steps of: decoding a singlestream to obtain a downsampled base signal and metadata indicative of asmoothed luminance signal through a standard codec for decoding encodeddata into standard dynamic range image data, a first dynamic range ofthe high-dynamic range image data higher than a second dynamic range ofthe standard dynamic range image data, the smoothed luminance signalindicative of a surplus of the first dynamic range of the high-dynamicrange image data over the second dynamic range of the standard dynamicrange image data; upsampling the downsampled base signal to obtain anunsampled base signal; converting the upsampled base signal representedin a YUV color space to a RGB color space; performing inverse adaptivemonotonic nonlinear mapping with data adaptive tuning to the upsampledbase signal to obtain the base signal, the base signal having a dynamicrange equal to the second dynamic range of the standard dynamic rangeimage data; obtaining a smoothed luminance signal based on the metadataindicative of the smoothed luminance signal and a luminance component ofthe upsampled based signal; and generating the high-dynamic range imagedata based on the base signal and the smoothed luminance signal.

In one or more embodiments, a method of generating image data isdisclosed. The method includes steps of: receiving a first stream;decoding the first stream to obtain a downsampled base signal; obtaininga regraded base signal based on the downsampled base signal; generatinga base signal by performing inverse adaptive monotonic nonlinear mappingwith data adaptive tuning to the regraded base signal; obtaining asmoothed luminance signal, the smoothed luminance signal indicative of asurplus in a dynamic range of the image data over the base signal; andgenerating the image data represented in a red-green-blue (RGB) colorspace based on the base signal and the smoothed luminance signal.

In one embodiment, a second stream different from the first stream canbe received. The smoothed luminance may be obtained by decoding thesecond stream.

In one embodiment, the smoothed luminance signal is obtained byobtaining metadata describing parameters for generating the smoothedluminance signal, and generating the smoothed luminance signal based onthe parameters.

Despite one or more embodiments disclosed herein relate to encoding anddecoding for high dynamic range image data, similar principles can beapplied to encoding and decoding high dynamic range video data.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an image communication system according toone embodiment.

FIG. 2A illustrates an image encoding system according to oneembodiment.

FIG. 2B illustrates an image encoding system according to anotherembodiment.

FIG. 2C illustrates an image encoding system according to anotherembodiment.

FIG. 3A illustrates a smoothed luminance generation module of the imageencoding system according to one embodiment.

FIG. 3B illustrates an example binned histogram of an image before dataadaptive tuning according to one embodiment.

FIG. 3C illustrates an example piecewise linear mapping of an imageaccording to the histogram in FIG. 3B.

FIG. 4A illustrates an image decoding system according to oneembodiment.

FIG. 4B illustrates an image decoding system according to anotherembodiment.

FIG. 4C illustrates an image decoding system according to anotherembodiment.

FIG. 5 illustrates a smoothed luminance generation module of the imagedecoding system according to another embodiment.

FIG. 6A is a flow chart illustrating a process of encoding high dynamicrange image data into a single stream according to one embodiment.

FIG. 6B is a flow chart illustrating a process of decoding a singlestream into high dynamic range image data according to one embodiment.

FIG. 7A is a flow chart illustrating a process of encoding high dynamicrange image data into two data streams according to one embodiment.

FIG. 7B is a flow chart illustrating a process of decoding two streamsinto high dynamic range image data according to one embodiment.

FIG. 8A is an example of a reconstructed sample image using aconventional high dynamic range encoding and decoding.

FIG. 8B is an example of a reconstructed sample image shown in FIG. 8Ausing enhanced high dynamic range encoding and decoding according to oneembodiment.

DETAILED DESCRIPTION

The figures and the following description describe certain embodimentsby way of illustration only. One skilled in the art will readilyrecognize from the following description that alternative embodiments ofthe structures and methods illustrated herein may be employed withoutdeparting from the principles described herein. Reference will now bemade in detail to several embodiments, examples of which are illustratedin the accompanying figures. It is noted that wherever practicablesimilar or like reference numbers may be used in the figures to indicatesimilar or like functionality.

EXAMPLE IMAGE ENCODING/DECODING SYSTEM

FIG. 1 is a block diagram of an image presentation system 100 accordingto one embodiment. In one embodiment, the image presentation system 100includes a source device 110, an image encoding system 130, an imagedecoding system 150, and a sink device 170. Each of these components canbe implemented in hardware, software, firmware, or a combinationthereof. In some embodiments, the source device 110, the image encodingsystem 130, the image decoding system 150, and the sink device 170 areimplemented in a single hardware system (e.g., a computing machine). Insome embodiments, one or more of the source device 110, the imageencoding system 130, the image decoding system 150, and the sink device170 are implemented as separate computing machines and connected througha cable. In alternative embodiments, different and/or additionalcomponents may be included or some components may be omitted. Inalternative embodiments, such components are connected through anetwork, such as Internet, satellite, cable, and other networks.

The source device 110 generates input image data 115 and transmits theinput image data 115 to the image encoding system 130. The imageencoding system 130 receives the input image data 115 from the sourcedevice 110, and encodes the input image data 115 to generate one or moredata streams 140. The image decoding system 150 receives the one or moredata streams 140 from the image encoding system 130, and decodes the oneor more data streams 140 to generate output image data 155. The sinkdevice 170 receives the output image data 155 and visually presents animage to a user.

The image encoding system 130 performs preprocessing on the input imagedata 115, which has a first dynamic range of an image component (e.g.,luminosity or a color gamut), and encodes the preprocessed input imagedata through an encoding process to generate an intermediate image data,which has a second dynamic range lower than the first dynamic range.Similarly, the image decoding system 150 receives the one or more datastreams 140, and decodes the one or more data streams 140 through animage decoding process, which regenerates the intermediate image datahaving the second dynamic range and generates the output image data 155,and then performs postprocessing on the decoded one or more data streamsto generate the output image data having the first dynamic range.Detailed structures and operations of the image encoding system 130 andthe image decoding system 150 are described in detail with respect toFIGS. 2 through 7 below.

In one embodiment, the input image data 115 and the output image data155 are high dynamic range image data capable of displaying luminosityof at least 1,000 nits and potentially up to 10,000 nits, while theintermediate image data are capable of displaying luminosity within amaximum allowable luminosity of the standard dynamic range image data(e.g., 100 nits). For example, the high dynamic range image dataconforms to BT.2020 standard, and the standard dynamic range image dataconforms to BT.709. In this embodiment, the image encoding system 130performs preprocessing on the input image data 115 to obtain theintermediate image data compatible with the standard dynamic range imagedata, and the image encoding system 130 performs encoding through aconventional encoding process through a standard codec (e.g., Main10High Efficiency Video Coding (HEVC)) to generate one or more datastreams 140. Similarly, the image decoding system 150 performs decodingthrough a conventional encoding process through a standard codec (e.g.,HEVC) on the one or more data streams 140 to reconstruct theintermediate image data compatible with the standard dynamic range imagedata, and performs postprocessing on the reconstructed intermediateimage data to generate the output image data 155.

In one embodiment, the image encoding system 130 transmits two datastreams 140 including a first data stream 140A and a second data stream140B to the image decoding system 150. In one embodiment, the first datastream 140A includes encoded data corresponding to a first image dataincluding color information having a lower dynamic range than a dynamicrange of the input image data 115, and the second data stream 140Bincludes encoded data corresponding to a second image data in grayscaleindicative of a surplus of the dynamic range of the input image data 115over the lower dynamic range of the first image data. Based on the firstimage data and the second image data received through the first datastream 140A and the second data stream 140B respectively, the imagedecoding system 150 can generate the output image data 155 having thedynamic range of the input image data 115.

In another embodiment, the image encoding system 130 transmits a singledata stream 140 including the first data stream 140A to the imagedecoding system 150 without transmitting the second data stream 140B. Inone embodiment, the second image data in the grayscale can be encodedand added to first data stream 140A. The image decoding system 150 candecode the first data stream 140A including the encoded first image dataand the encoded second image data, and generates the output image data155 based on the decoded first image data and the decoded second imagedata obtained through the single data stream 140. Yet in anotherembodiment, the image encoding system 130 adds metadata to the firstdata stream 140A. The metadata include information or parameters usedfor generating the second image data at the image encoding system 130.Based on the metadata and the first image data obtained through thefirst data stream 140A, the image decoding system 150 can generate (orinfer) the second image data, and further generate the output image data155 based on the first image data and the second image data.Accordingly, the second data stream 140B need not be transmitted, thusbandwidth of information exchanged between the image encoding system 130and the image decoding system 150 can be conserved.

FIG. 2A illustrates an image encoding system 130A according to oneembodiment. In this embodiment, the image encoding system 130A receivesthe input image data 115 and generates a single data stream 140. Thedata stream 140 is a stream of bits corresponding to a signal or imagedata having a lower dynamic range of an image component (e.g.,luminosity or a color gamut) than a dynamic range of the input imagedata 115. In one aspect, the image encoding system 130A implements thecolor conversion module 210, a smoothed luminance (SL) generation module220, a base signal generation module 230, a data adaptive tuning (DAT)module 240, a color conversion module 250, a downsampling module 260,and an encoder 270. These components can be implemented as hardware,software, firmware, or a combination thereof. In alternativeembodiments, different and/or additional components may be included orsome components may be omitted.

The color conversion module 210 receives the input image data 115 andextracts a luminance component of the input image data 115. In oneaspect, the input image data 115 is a high dynamic range image datarepresented in the RGB color space. The color conversion module 210performs color conversion to convert the input image data represented inthe RGB color space into a YUV color space, and then outputs a luminancecomponent 215 of the converted image data. In one example, the YUV colorspace is a YCbCr color space, a YFbFr color space, or some derivationthereof (e.g., Y′CbCr). The YFbFr color space is a color space obtainedthrough a number of lossless integer color transforms, using liftingsteps, which are especially effective and efficient to implement.Details of the YFbFr color space are disclosed in U.S. patentapplication Ser. No. 14/226,680 (“ACTED”), and U.S. Pat. No. RE40,081,which are incorporated herein in their entirety.

In one embodiment, the color transform may be adaptive to input data,with the adaptation taking place at the sequence, GOP, frame, slice, oreven block-level. The selection can be made for example by measuring thedecorrelation efficiency on the input data (according to a measure suchas coding gain, or by actually coding the data and using rate-distortionanalysis). If the color transforms are selected from a small table, onecan signal the selection by a simple index in the metadata in thebitstream.

The SL generation module 220 receives the luminance component 215 fromthe color conversion module 210 and generates the SL signal 225. In oneaspect, the smoothed luminance (SL) signal 225 is an intensity-onlysignal (or grayscale image data), which in some embodiments contains thelow frequency intensity information of the input image data 115, or agraded version of the luminance. In some embodiments, the SL signal 225is generated by performing non-linear mapping on the luminance component215. In one example, as depicted in FIG. 3A, the SL generation module220 applies a series of operations to the luminance component 215including: applying an OETF, applying a monotonically increasingnonlinear function, applying an EOTF. Such a combination of operationspermits creating a method to regrade the initial high dynamic rangeimage data into standard dynamic range data (or more generally, creatingimage data with a different peak luminosity), while working in aperceptually more uniform domain. This allows the regrading to appearmore natural and consistent. Components of this process, or the entireprocess, may be computed by a piecewise polynomial function of finitesteps.

The base signal generation module 230 receives the input image data 115and the SL signal 225, and generates the base signal 235. The basesignal 235 is a signal or image data having a lower dynamic range thanthe input image data. In one aspect, the base signal 235 is compatiblewith the standard dynamic range image data capable of representing up to100 nits, hence the base signal 235 may be displayed through by astandard dynamic range display device. In the embodiment shown in FIG.2A, the base signal generation module 230 generates the base signal 235represented in the RGB color space. Specifically, the base signalgeneration module 230 generates the base signal 235 by dividing theinput image data 115 represented in the RGB color space by the SL signal225. That is, each color component (RGB) of the base signal 235 isdivided by the SL signal 225. Hence, the input image data 115 can berepresented as the following Equation (1):RGB _(HDR)=SL*B   (1)where RGB_(HDR) corresponds to values of the input image data 115 (forred, green, blue components), SL corresponds to the SL signal 225, and Bcorresponds to the base signal 235 of red, green, blue color components.In one aspect, the SL signal 225 represents a surplus of the firstdynamic range of the input image data 115 over the second dynamic rangeof the lower dynamic range image data or the base signal.

The DAT module 240 receives the base signal 235 from the base signalgeneration module 230, and performs adaptive monotonic nonlinear mappingwith data adaptive tuning to the base signal 235 to generate a regradedbase signal 255. In the embodiment shown in FIG. 2A, the DAT module 240receives the base signal 255 in the RGB color space, and performsadaptive monotonic nonlinear mapping with data adaptive tuning in theRGB color space to generate the regraded base signal 245 in the RGBcolor space. In some embodiments, the regraded base signal 255 isgenerated by performing a nonlinear mapping on the base signal 235, andapplying data-adaptive tuning. In one example, the DAT module 240applies a monotonically increasing nonlinear function to the base signal235 to improve the naturalness and consistency of the regraded imagerywith respect to the higher luminosity signal. In one approach, the DATmodule 240 measures statistics on the base signal 255 (e.g., a histogramof the luminosity component), and performs adaptive monotonic nonlinearmapping based on the measured statistics of the base signal 255. In someembodiments, the data adaptive tuning performed by the DAT module 240can be implemented in a similar manner as the SL generation module 220.

The color conversion module 250 receives the regraded base signal 255represented in the RGB color space, and performs color conversion toobtain the regraded base signal 255 represented in the YUV color space.In one embodiment, the color conversion modules 210 and 250 perform asame type of color conversion. Alternatively, the color conversionmodule 210 may extract only the luminance component 215, while the colorconversion module 250 converts all image components (e.g., RGB values)in one color space into all image components (e.g., YUV values) inanother color space.

The downsampling module 260 receives the regraded base signal 255 anddownsamples the regraded base signal 255 to generate a downsampled basesignal 265. In the embodiment shown in FIG. 2A, the downsampling module260 receives the regraded base signal 255 represented in the YUV colorspace, and performs downsampling in the YUV color space to generate thedownsampled base signal 265 represented in the YUV color space. In oneembodiment, the downsampling module 260 downsamples the chromacomponents (i.e., color components) of the regraded base signal 255, butnot the luminance component of the regraded base signal 255. In oneembodiment, the downsampling module 260 downsamples the regraded basesignal 255 in a YUV 4:4:4 format to a YUV 4:2:2 format (i.e.,downsamples horizontally, but not vertically). In another embodiment,the downsampling module 260 downsamples to the regraded base signal 255in the YUV 4:4:4 format to the YUV 4:2:0 format (i.e., it downsamples inboth directions).

Note that in some embodiments, the DAT module 240 may be interspersed inbetween these conversions; that is, the DAT module 240 may be applied inany of the domains: YUV 4:4:4, YUV 4:2:2, or YUV 4:2:0. Likewise, theencoder 270 may be capable of encoding any of YUV 4:4:4, YUV 4:2:2, orYUV 4:2:0.

In one embodiment, the filters implemented by the downsampling module260 are adaptive to input data, with the adaptation taking place at thesequence, GOP, frame, slice, or even block-level. The selection can bemade, for example, by measuring the reconstruction fidelity (accordingto a measure such as peak signal-to-noise ratio (PSNR), with many othersavailable) of the signal after consecutively down and up sampling, or byactually coding the data and using rate-distortion analysis. If theresampling filters are selected from a small table, the selection can beindicated by a simple index in the metadata in the data stream 140.

In one implementation, the downsampling module 260 includes one or morefilters to downsample the regraded base signal 255. Given a discretefilter x[n], M unique phases of filter x can be obtained by downsamplingx by M. For example, when M=2, there are two phases, which can belabeled as the 0-phase and the 1-phase; the 0-phase of the discretefilter x[n] is x[2n], and the 1-phase of x[n] is x[2n+1]. In thisconfiguration, the other phases can be derived from the zero phase bynon-unique interpolation. As a specific example, a standard grid forresampling in chroma from a 4:4:4 format to a 4:2:0 format, as practicedin both AVC and HEVC, requires the use of 4-phase resampling. As aconcrete example, suppose the zero-phase 2N+1 tap horizontal downsampling filter is hd[n], where n=−N, . . . , −1, 0, 1, . . . N. Thenthe 1-phase vertical down sampling filter vd[n] can be derived byinterpolating hd[n] by a factor of 2 using spline interpolation methodto get d[n], where hd[n]=hd[2n], for n=−N, . . . , −1, 0, 1, . . . N.The horizontal down sampling filter is the 0-0phase filter of hd2[n],and the vertical down sampling filter is the 1-phase filter of hd2[n],where vd[n]=hd2[2n+1], for n=−N, . . . , −1, 0, 1, . . . N. Examples offilter values are provided in Table 1 below.

TABLE 1 Example Filter Values Hd (+8.0, 0.0, −64.0, +128.0, +368.0,+128.0, −64.0, 0.0, +8.0)/256 D ((+8.0, +8.0, 0.0, 0.0, −24.0,−64.0,+48.0, +128.0, +224.0, +368.0, +224.0, +128.0, +48.0, −64.0, −24.0, 0.0,0.0, +8.0, +8.0)/256 Vd (+8.0, 0.0, −24.0, +48.0, +224.0, +224.0, +48.0,−24.0, +0.0, +8.0)/ 256

In one embodiment, the downsampling module 260 includes a horizontaldownsampling filter for downsampling the regraded base signal 255 in the4:4:4 format down to a 4:2:2 format, and a vertical downsampling filterfor downsampling the downsampled signal in the 4:2:2 format down to the4:2:0 format. The vertical downsampling filter may be a phase shiftedversion of the horizontal downsampling filter. Hence, the regraded basesignal 255 represented in the 4:4:4 format can be downsampled to obtainthe downsampled base signal represented the 4:2:0 format. Examplecoefficients of the horizontal downsampling filter is listed in Table 2,and example coefficients of the vertical downsampling filter is listedin Table 3.

TABLE 2 Example Horizontal Downsampling Filter Coefficients (4:4:4 to4:2:2) Tap k −5 −4 −3 −2 −1 0 1 2 3 4 5 Coefs c1[k] 1 0 −3 0 10 16 10 0−3 0 1

TABLE 3 Example Vertical Downsampling Filter Coefficients (4:2:2 to4:2:0) Tap k −5 −4 −3 −2 −1 0 1 2 3 4 5 6 Coefs c2[k] 9 13 −27 −42 75228 228 75 −42 −27 13 9

The encoder 270 receives the downsampled base signal 265 from thedownsampling module 260, and generates an encoded base signal 275A. Theencoder 270 can implement a standard codec (e.g., ITU/ISO/IECHEVC/H.265, ITU/ISO/IEC AVC/H.264, ITU/ISO/IEC MPEG-2/H.262, Alliancefor Open Media, Google VPX, and Xiph Theora, etc.) for encoding standarddynamic range image data. The encoder 270 can also implement othercodecs for encoding image data having a lower dynamic range than adynamic range of the input image data 115. In one embodiment, theencoder 270 generates the data stream 140 by encoding the downsampledbase signal 265 using the codec and adding metadata in the supplementalenhancement information (SEI) 228 indicative of parameters used in theSL generation module 220 for generating the SL signal 225, and alsometadata describing parameters used in the DAT module 240 for generatingthe regraded base signal 245 to the encoded downsampled base signal 265.By integrating the SEI 228 with the encoded base signal, bandwidth ofinformation transmitted to the image decoding system 150 can be greatlyreduced, while still allowing full reconstruction of the high dynamicrange data.

In alternative embodiment, an additional encoder is implemented in theimage encoding system 130A to encode the SL signal 225. The encoded SLsignal can be combined with the encoded base signal again as metadata inan SEI, and transmitted to the image decoding system 150 through thesingle data stream 140. This requires only a slightly higher bandwidth(about 1% higher) then a method in which the SL is derived, but providesfor a more precise recovery of the SL signal, and reconstruction of theHDR signal.

FIG. 2B illustrates an image encoding system 130B according to anotherembodiment. The image encoding system 130B receives the input image data115 and generates the data stream 140. In the embodiment shown in FIG.2B, the encoding system 130B includes a single color conversion module212, the SL generation module 220, the base signal generation module230, the DAT module 240, the downsampling module 260, and the encoder270. The configurations of these components are similar to the onesshown in FIG. 2A, except that a single color conversion module 212 isimplemented instead of the color conversion modules 210, 250 of FIG. 2A.

Specifically in the embodiment shown in FIG. 2B, an output of the colorconversion module 212 is directly coupled to an input of the base signalgeneration module 230 and an input of the SL generation module 220.Hence, the base signal generation module 230 receives the input imagedata 218 represented in the YUV color space, and obtains the base signal238 represented in the YUV color space. In one aspect, the base signal238 can be obtained according to the following Equation (2):

$\begin{matrix}{{Y_{SDR} = \frac{Y_{HDR}}{Y\_{SL}}},{{Cb_{SDR}} = \frac{Cb_{HDR}}{{Cb}\_{SL}}},{{Cr_{SDR}} = \frac{{Cr}_{HDR}}{{Cr}\_{SL}}}} & (2)\end{matrix}$where Y_(HDR) is a luminance component of the input image data 218represented in the YUV color space, Cb_(HDR) is a blue difference chromacomponent of the input image data 218 represented in the YUV colorspace, Cr_(HDR) is a red difference chroma component of the input imagedata 218 represented in the YUV color space. In this embodiment, theY_SL, Cb_SL, and Cr_SL, are new components that can be functionallyderived from the previously computed SL signal; this is designed toallow a slightly difference regrading in chroma than luma if desired; ifall are set to equal SL, this reduces to the previous approach. Y_SL canbe a luminance component of the SL signal 225, Cb_SL can be a derivedblue difference chroma component of the SL signal 225, and Cr_SL can bea derived red difference component of the SL signal 225. Y_(SDR) is aluminance component of the base signal 238 represented in the YUV colorspace, Cb_(SDR) is a blue difference chroma component of the base signal238 represented in the YUV color space, and Cr_(SDR) is a red differencechroma component of the base signal 238 represented in the YUV colorspace. In some embodiments, the Cb_SL and Cr_SL can be replaced withY_SL as before.

The remaining components (e.g., the SL generation module 220, thedownsampling module 260, the DAT module 240, and the encoder 270)operate as shown in FIG. 2A. Therefore, the detailed description thereofis omitted herein for the sake of brevity.

FIG. 2C illustrates an image encoding system 130C according to anotherembodiment. In this embodiment, the image encoding system 130C receivesthe input image data 115 and generates dual data streams 140. In theembodiment shown in FIG. 2C, the encoding system 130B includes the colorconversion module 212, the SL generation module 220, the base signalgeneration module 230, the DAT module 240, the downsampling module 260,and encoders 270A, 270B. The configurations of these components aresimilar to the ones shown in FIG. 2B, except that two encoders 270A,270B generate two data streams 140A, 140B. These two streams can becombined into one stream according to a scalable coding architecture, orremain as two separate streams.

The encoder 270A receives the downsampled base signal 265 from thedownsampling module 260, and generates the data stream 140A. In oneembodiment, the encoder 270A implements a standard codec (e.g.,ITU/ISO/IEC HEVC/H.265, ITU/ISO/IEC AVC/H.264, ITU/ISO/IEC MPEG-2/H.262,Alliance for Open Media, Google VPX, and Xiph Theora, etc.) for encodingstandard dynamic range image data. In some embodiments, the encoder 270Aimplements other codecs for encoding image data having a lower dynamicrange than a dynamic range of the input image data 115.

The encoder 270B receives the SL signal 225 from the SL generationmodule 220 and encodes the SL signal 225 to generate the data stream140B. In one implementation, the data stream 140B includes encoded SLsignal. In one embodiment, the encoder 270B implements the same standardcodec implemented by the encoder 270A. In another embodiment, theencoder 270B implements a codec customized for encoding the SL signal225 different from the standard codec implemented by the encoder 270A.

FIG. 3A illustrates the SL generation module 220 of the image encodingsystem 130 according to one embodiment. In one embodiment, the SLgeneration module 220 includes an OETF module 310, a tone mapping module320, and an electro-optical transfer function (EOTF) module 330. Thesecomponents operate together to receive the luminance component 215 andgenerate the SL signal 225 based on the luminance component 215. In oneaspect, the SL generation module 220 applies adaptive techniques forgenerating and scaling the SL signal 225. These components can beimplemented as hardware, software, firmware, or a combination thereof.In alternative embodiments, different and/or additional components maybe included or some components may be omitted.

The OETF module 310 receives the luminance component 215 and applies atransfer function for transforming the luminance component 215represented in a linear luminance domain into a non-linear luminancedomain prior to performing non-linear mapping. In one embodiment, theOETF module 310 applies a transfer function such as the SMPTE (Societyfor Motion Picture and Television Engineers) standards ST-2084 (PQ TF)or the Philips TF (potentially part of ST-2094.20). In anotherembodiment, the OETF module 310 applies a transfer function as shown inEquation (3) below:Y _(OETF)=log((Rho−1)*powf(Y,(1/2.40))+1)/log(Rho),   (3)where ho=32.0*powf((InputPeakBrightness/TgtPeakBrightness), (1/2.40))+1.

The InputPeakBrightness is a peak brightness (e.g., 1,000 nits) of theinput image data 115 or the luminance component 215 automaticallymeasured, for example by the OETF module 310. The TargetPeakBrightnessis a target peak brightness is a parameter in the transfer process, andmay for example be selected as the desired output peak brightness value(e.g., 100 nits, starting with 1000 nits input). By measuring theInputPeakBrightness, the transformation of the luminance component 215can be performed in an adaptive manner, to achieve a more natural,data-appropriate transfer function.

Yet in another embodiment, the OETF module 310 applies a transferfunction as shown in Equation (4) below:

$\begin{matrix}{{Y_{OETF} = {{powf}\left( {\left( {\left( {{{c2}*{{powf}\left( {Y,{m1}} \right)}} + {c1}} \right)/\left( {1.0 + {{c3}*{{powf}\left( {Y,{m1}} \right)}}} \right)} \right),{m2}} \right)}},\mspace{79mu}{{{where}\mspace{14mu}{{powf}\left( {a,b} \right)}} = a^{b}},{{m1} = \left( {{2610.0/4096.0}*4.0} \right)},{{m2} = {\left( \frac{\left( {2523.0*128.0} \right)}{4096} \right)*\left( {1 + {0.25*{\log\left( \frac{TgtPeakBrightness}{InputPeakBrightness} \right)}}} \right)}},{{c1} = {(3424.0)/4096.0}},{{c2} = {\left( {2413.0*32.0} \right)/4096.0}},{{{and}\mspace{14mu}{c3}} = {\left( {2392.0*32.0} \right)/4096.0}}} & (4)\end{matrix}$

In addition, Data-Adaptive Tuning (DAT) can be applied to the transferfunction, which uses statistics of the data to improve coding. Byapplying DAT, the OETF module 310 generates the transformed luminancesignal Y_OETF 315. In some embodiments, both the method of tuning (e.g.,which TF function to use, and whether additional nonlinear functions areapplied), and the frequency of tuning, whether applied at the sequence,GOP, frame-level, or even slice or block-level may be varied incomplexity.

The tone mapping module 320 receives the transformed luminance signalY_OETF 315 from the OETF module 310 and performs non-linear mapping onthe transformed luminance signal Y_OETF 315. Specifically, the tonemapping module 320 controls the dynamic range of the input and theoutput of the tone mapping. For example, the tone mapping module 320applies a polynomial mapping represented as a power function shown inEquation (5) below:y=(a*x+b)^(∝),   (5)where x corresponds to transformed luminance signal Y_OETF 315, and y isthe mapped luminance signal Y′_OETF 325. For simplicity, apiecewise-linear model with L pieces can be applied, where L>=1. Foreach piece, Equation (6) reduces to:y=a(k)*x+b(k),k=0,1, . . . ,(L−1)   (6)where, k represents the k^(th) piece or bin. These equations are appliedto the transformed luminance signal Y_OETF 315 as input x to generatethe mapped luminance signal Y′_OETF 325.

In one embodiment, the coefficients a(k), b(k) in the Equation (6) aboveare sequence dependent and are derived based on the distributioncharacteristics of the HDR input signal. In one example, the tonemapping module 320 determines the coefficients a(k), b(k) based on ahistogram analysis. For example, the transformed luminance signal Y_OETF315 is segmented to L bins, and a count is obtained for the number ofpixels in each bin (which we call binsize). FIG. 3B illustrates anexample luminosity histogram of a luminance image before data adaptivetuning, where a luminance of transformed luminance signal Y_OETF 315 isdivided into an example 8 bins. Under an optimization process, thenumber of bins and their respective sizes are automatically determinedaccording to an input data such that the lower and mid tone levels havean improved representation compared to the unadjusted transfer functionrepresentation. Thus, the coefficients a(k), b(k) can be determined asshown in Equation (7) below:

$\begin{matrix}{{{a(k)} = {\left( \frac{{binsize}_{k}}{NumTotPixels} \right)*\left( {{MAX}_{-}{DISPLAY}_{-}L{{UMA}/{MAX}_{-}}LUMA_{-}{BIN}_{k}} \right)}}{{b(0)} = 0}{{b(k)} = {{{a\left( {k - 1} \right)}*{MAX}_{-}LUMA_{-}{BIN}_{k - 1}} + {b\left( {k - 1} \right)} - {{a(k)}*{MAX}_{-}LUMA_{-}{BIN}_{k - 1}}}}} & (7)\end{matrix}$where MAX_LUMA_BIN_(k) is the maximum luminance value of the transformedluminance signal Y_OETF 315 in bin k, and binsize_(k) is the number ofpixels in bin k. The piecewise linear DAT function corresponding to FIG.3B is given in FIG. 3C. In one approach, the numbers binsize_(k) aresent in the data stream 140 as metadata (e.g., SEI 275B of FIG. 2C). Inanother approach, these numbers are indexed, and only an index is sentfor each number. In one embodiment, this DAT can easily be applied atthe sequence, GOP, frame, slice, or block level.

The EOTF module 330 receives the mapped luminance signal Y′_OETF 325from the tone mapping module 320, and generates the SL signal 225. Inone embodiment, the EOTF module 330 converts the mapped luminance signalY′_OETF 325 represented in a perceptual luminance domain into a linearluminance domain. In one example, the EOTF module 330 obtains the SLsignal 225 according to Equation (8) below:SL=Y/TL,   (8)where SL corresponds to the SL signal 225, Y corresponds to theluminance component 215, and TL is an intermediate signal for generatingthe SL signal 225 based on the luminance component 215. In case the OETFmodule 310 implements the transfer function as shown in equation (3),the EOTF module 330 produces the intermediate signal TL as shown in anequation (9) below:TL=powf((powf(Rho,Y′_(OETF))−1)/(Rho−1),2.40).   (9)In case the OETF module 310 implements the transfer function as shown inEquation (4), the EOTF module 330 produces a TL as shown in Equation(10) below:

$\begin{matrix}{{TL} = {{{powf}\left( {\frac{{Max}\left( {{0.0},{p{{{owf}\left( {Y_{OETF}^{\prime},\frac{1.0}{m2}} \right)} \cdot {c1}}}} \right)}{{c2} - {{c3}*{{powf}\left( {Y_{OETF}^{\prime},\frac{1.0}{m2}} \right)}}},\ \frac{1.0}{m1}} \right)}.}} & (10)\end{matrix}$where Max(a,b) returns a if a>b, else b.

In one embodiment, the SL generation module 220 obtains the SL signal225 by a smoothing process on the input luminance component 215. In oneembodiment, the SL signal 225 is obtained as a linear combination of apre-defined shaped function. As an example, a 2D Gaussian function canbe employed, because the impulse response of many natural and manmadesystems is Gaussian. The individual shape functions are thenmathematically built as a partition of unity. In another embodiment, theSL signal 225 is obtained by applying a smoothing filter (again such asby a 2D Gaussian) to the luminance component 215.

Modeling of the smoothed luminance signal by separable shape functionsis challenging, as is smoothing a signal by a Gaussian function. Forexample, the modeling involves matrix computations, which are nothardware friendly and requires considerable memory; Gaussian smoothingis likewise not hardware friendly. To overcome those disadvantages, amoving average filter can be used to obtain the low frequency version ofthe image. The conventional coding techniques are too complicated toperform. The moving average filter is hardware friendly and easy toimplement as an add-on or pre/processing unit.

Taking M point moving averaging filter as an example, where M is chosenas an odd number, and the input signal is symmetrical around the outputsignal. Let M=2N+1, the M point moving average filter is defined asEquation (11) below:

$\begin{matrix}{{y\lbrack i\rbrack} = {\frac{1}{M}{\sum\limits_{j = {- N}}^{N}{x\left\lbrack {i + j} \right\rbrack}}}} & (11)\end{matrix}$By analysis using the Central Limit Theorem, a high qualityapproximation of Gaussian smoothing can be obtained by the simple movingaverage filter.

FIG. 4A is an image decoding system 150A according to one embodiment.The image decoding system 150A receives a single data stream 140, forexample, from the image encoding system 130A of FIG. 2A, and generatesthe output image data 155. In one aspect, the output image data are aHDR image data represented in the RGB color space. In one embodiment,the image decoding system 150A includes a decoder 420, an upsamplingmodule 430, a color conversion module 440, an inverse DAT module 450, anenhanced image generation module 460, and a SL generation module 480.These components can be implemented as hardware, software, firmware, ora combination thereof. In alternative embodiments, different and/oradditional components may be included or some components may be omitted.

The decoder 420 receives the input data stream 140, for example, fromthe image encoding system 130A, and generates a regraded downsampledbase signal 425. The decoder 420 decodes the input data stream 140 togenerate the regraded downsampled base signal 425 in the YUV colorspace. In one embodiment, the decoder 420 implements a standard codec(e.g., ITU/ISO/IEC HEVC/H.265, ITU/ISO/IEC AVC/H.264, ITU/ISO/IECMPEG-2/H.262, Alliance for Open Media, Google VPX, and Xiph Theora,etc.) for decoding encoded data into standard dynamic range image data.In some embodiments, the decoder 420 implements other codecs fordecoding encoded data into lower dynamic range image data having a lowerdynamic range than a dynamic range of the output image data 155.

In one embodiment, the decoder 420 receives the input data stream 140,and obtains the SEI (e.g., InputPeakBrightness, TargetPeakBrightness, anumber of bins, binsize_(k), etc.) by decoding a portion of the inputdata stream 140 corresponding to metadata.

The SL generation module 480 receives the regraded downsampled basesignal 425 and the SEI from the decoder 420, and generates the SL signal485. In one embodiment, the SL generation module 480 obtains informationdescribing the DAT performed by the image encoding system 130 in theSEI, and applies inverse DAT to the regraded downsampled base signal 425to infer the SL signal 485 without actually receiving the SL signal fromthe image encoding system 150A. Detailed structure and operation of theSL generation module 480 are described below with respect to FIG. 5.

The upsampling module 430 receives the regraded downsampled base signal425 from the decoder 420A, and generates a regraded base signal 435. Inthe embodiment shown in FIG. 4A, the upsampling module 430 receives theregraded downsampled base signal 425 represented in the YUV color space,and performs upsampling in the YUV color space to generate the regradedbase signal 435 represented in the YUV color space. In one embodiment,the upsampling performed by the upsampling module 430 is inverse to thedownsampling performed by the downsampling module 260 of FIG. 2A.

In one embodiment, the upsampling module 430 implements interpolationfilters for generating the regraded base signal 435. The filtersimplemented by the upsampling module 430 may be adaptive to input data,with the adaptation taking place at the sequence, GOP, frame, slice, oreven block-level.

In one embodiment, the upsampling module 430 includes four interpolationfilters to upsample the regraded downsampled base signal 425. Forexample, horizontal up sampling filter for even pixels, vertically upsampling filter for odd pixels, horizontal up sampling filter for oddpixels, and vertically up sampling filter for even pixels can beimplemented. Let u[n] be a (non-unique) discrete interpolation filter,and then the filters used for horizontal and vertical upsampling can beidentified as Equation (12) below:hue[n]=u[4n], vuo[n]=u[4n+1],huo[n]=u [4n+2], and vue[n]=u[4n+3].   (12)Example filter values are provided in Table 4 below.

TABLE 4 Example Filter Values u (+8.0, +0.0, −16.0, −16.0, −32.0, +0.0,+56.0, +144.0, +240.0, +256, +240.0, +144.0, +56.0, +0.0, −32.0, −16.0,−16.0, hue (+256.0)/256 vuo (+0.0, −16.0, +56.0, +240.0, −32.0,+8.0)/256 huo (−16.0, +144.0, +144.0, −16.0) /256 vue (+8.0, −32.0,+240.0, +56.0, −16.0, +0.0)/256

As an example, two separate vertical upsampling filters are used torestore samples at odd and even indices, respectively. Example filtercoefficients for even samples filter are provided in Table 5, whilethose for the odd samples are provided in Table 6. Similarly, twoseparate horizontal upsampling filters are used to restore samples atodd and even indices, respectively. Example filter coefficients for evensamples filter are provided in Table 7, while those for the odd samplesare provided in Table 8.

TABLE 5 Example Vertical Upsampling Filter Coefficients (4:2:0 to 4:2:2)for Even Samples Tap k −3 −2 −1 0 1 2 Coefs d1[k] 9 −27 75 228 −42 13

TABLE 6 Example Vertical Upsampling Filter Coefficients (4:2:0 to 4:2:2)for Odd Samples Tap k −2 −1 0 1 2 3 Coefs d2[k] 9 13 −27 −42 75 228

TABLE 7 Example Horizontal Upsampling Filter Coefficients (4:2:2 to4:4:4) for Even Samples Tap k  0 Coefs d3[k] 16

TABLE 8 Example Horizontal Upsampling Filter Coefficients (4:2:2 to4:4:4) for Odd Samples Tap k −2 −1 0 1 2 3 Coefs d4[k] 1 −3 10 10 −3 1

The color conversion module 440 receives the regraded base signal 435from the inverse DAT module 450 and performs color conversion. In theembodiment shown in FIG. 4A, the color conversion module 440 receivesthe regraded base signal 435 represented in the YUV color space, andperforms color conversion to obtain the regraded base signal 445represented in the RGB color space.

The inverse DAT module 450 receives the regraded base signal 445 fromthe color conversion module 440, and applies an inverse transferfunction to the regraded base signal 445 to obtain a base signal 455. Inthe embodiment shown in FIG. 4A, the inverse DAT module 450 receives theregraded base signal 445 in the RGB color space, and performs inverseadaptive monotonic nonlinear mapping with data adaptive tuning in theRGB color space to generate the base signal 455 in the RGB color space.In some embodiments, the base signal 455 is generated by performing aninverse process to the adaptive monotonic nonlinear mapping with dataadaptive tuning performed by the DAT module 240 of FIG. 2A. For example,the parameters used for performing DAT can be extracted from themetadata included in the input data stream 140, and the inverse adaptivemonotonic nonlinear mapping can be performed by the inverse DAT module450 according to the extracted parameters. In some embodiments, inverseadaptive monotonic nonlinear mapping, and/or data adaptive tuningperformed by the inverse DAT module 450 can be implemented in a similarmanner as the SL generation module 480.

The enhanced image generation module 460 receives the base signal 455from the inverse DAT module 450 and the SL signal 485 from the SLgeneration module 480, and generates the output image data 155. In theembodiment shown in FIG. 4A, the enhanced image generation module 460receives the base signal 455 represented in the RGB color space andobtains the output image data 155 represented in the RGB color spacebased on the base signal 455 and the SL signal 422. Specifically, theenhanced image generation module 460 generates the output image data 155by multiplying the base signal 455 represented in the RGB color space bythe SL signal 422. That is, each color component (RGB) of the basesignal 455 is multiplied by the SL signal 422. Hence, the output imagedata 155 can be represented as the following Equation (13):RGB_(HDR) =SL*B  (13)where RGB_(HDR) corresponds to values of the output image data 155 (forred, green, blue components), SL corresponds to the SL signal 485, and Bcorresponds to the base signal 455 of red, green, blue color components.The output image data 155 can be provided to the sink device 170 of FIG.1, and displayed to the user.

FIG. 4B illustrates an image decoding system 150B according to anotherembodiment. The image decoding system 150B receives a single input datastream 140, for example, from the image encoding system 130B shown inFIG. 2B. In the embodiment shown in FIG. 4B, the image decoding system150B includes the decoder 420, the upsampling module 430, the colorconversion module 440, the inverse DAT module 450, the SL generationmodule 480, and the enhanced image generation module 460. Theconfigurations of these components are similar to the ones shown in FIG.4A, except that the color conversion module 440 between the upsamplingmodule 430 and the inverse DAT module 450 in FIG. 4A is provided afterthe enhanced image generation module 460 in FIG. 4B. Accordingly, theinverse DAT module 450 and the enhanced image generation module 460operate in the YUV color space. The decoder 420, the SL generationmodule 480, and the upsampling module 430 operate as shown in FIG. 4A.Thus, the detailed description thereof is omitted herein for the sake ofbrevity.

In the embodiment shown in FIG. 4B, the inverse DAT module 450 receivesthe regraded base signal 435 represented in the YUV color space, andobtains the base signal 458 represented in the YUV color space. In thisembodiment, the inverse DAT module 450 performs inverse adaptivemonotonic nonlinear mapping with data adaptive tuning in the YUV colorspace, instead of the RGB color space.

The enhanced image generation module 460 receives the base signal 458represented in the YUV color space from the inverse DAT module 450, andthe SL signal 485 from the SL generation module 480 to generate theoutput image data 468 represented in the YUV color space. In oneembodiment, the enhanced image generation module 460 obtains the outputimage data 468 according to Equation (14) below:Y _(HDR) =Y _(SDR) *Y_SL, Cb _(HDR) =Cb _(SDR) *Cb_SL, Y _(HDR) =Cr_(SDR) *Cr_SL   (14)where Y_(HDR) is a luminance component of the output image data 468represented in the YUV color space, Cb_(HDR) is a blue difference chromacomponent of the output image data 468 represented in the YUV colorspace, Cr_(HDR) is a red difference chroma component of the output imagedata 468 represented in the YUV color space, Y_SL can be a luminancecomponent of the SL signal 485, Cb_SL can be a derived blue differencechroma component of the SL signal 485, and Cr_SL can be a derived reddifference component of the SL signal 485. Y_(SDR) is a luminancecomponent of the base signal 458 represented in the YUV color space,Cb_(SDR) is a blue difference chroma component of the base signal 458represented in the YUV color space, and Cr_(SDR) is a red differencechroma component of the base signal 458 represented in the YUV colorspace. In some embodiments, the Cb_SL and Cr_SL can be replaced withY_SL.

In the embodiment shown in FIG. 4B, the color conversion module 440receives the output image data 468 represented in the YUV color space,and performs color conversion to obtain the output image data 155represented in the RGB color space.

FIG. 4C illustrates an image decoding system 150C according to anotherembodiment. The image decoding system 150C receives dual input datastreams 140A and 140B, for example, from the image encoding system 130Cshown in FIG. 2C. In the embodiment shown in FIG. 4C, the image decodingsystem 150B includes the decoders 420A, 420B, the upsampling module 430,the color conversion module 440, the inverse DAT module 450, and theenhanced image generation module 460. The configurations of thesecomponents are similar to the ones shown in FIG. 4B, except that the SLgeneration module 480 of FIG. 4B is omitted and an additional decoder420B is provided to obtain the SL signal 485. The decoder 420A, theupsampling module 430, the inverse DAT module 450, the enhanced imagegeneration module 460, and the color conversion module 440 operate asshown in FIG. 4B. Thus, the detailed description thereof is omittedherein for the sake of brevity.

In the embodiment shown in FIG. 4C, the decoder 420B receives the inputdata stream 140B including an encoded SL signal, and decodes the inputdata stream 140B to obtain the SL signal 485. In one embodiment, thedecoder 420B implements a standard codec (e.g., ITU/ISO/IEC HEVC/H.265,ITU/ISO/IEC AVC/H.264, ITU/ISO/IEC MPEG-2/H.262, Google VPX, and XiphTheora, etc.) for decoding encoded data into standard dynamic rangeimage data. In some embodiments, the decoder 420B implements othercodecs for decoding encoded data into lower dynamic range image datahaving a lower dynamic range than a dynamic range of the output imagedata 155.

In this embodiment shown in FIG. 4C, a simple decoding process isperformed by the decoder 420B, without implementing the SL generationmodule 480 of FIG. 2 to infer the SL signal 485. Hence, the computingresource of the image decoding system 150C can be reduced compared tothe image decoding system 150C of FIG. 4B, at the cost of additionalbandwidth for receiving the input data stream 140B.

FIG. 5 is a smoothed luminance generation module 480 of the imagedecoding system 150A or 150B, according to another embodiment. The SLgeneration module 480 receives the regraded downsampled base signal 425and parameters used for performing adaptive tone mapping and/or DAT bythe image encoding system 130 and generates (or infers) the SL signal485. In one embodiment, the SL generation module 480 includes a lumaextraction module 510, an inverse tone mapping module 520, and a SLoutput module 530. These components can be implemented as hardware,software, firmware, or a combination thereof. In alternativeembodiments, different and/or additional components may be included orsome components may be omitted.

The luma extraction module 510 receives the regraded downsampled basesignal 425 from the decoder 420, and obtains a luminance componentY′_SDR 515 of the regraded downsampled base signal 425.

The inverse tone mapping module 520 receives parameters used forperforming adaptive tone mapping and/or DAT from the decoder 420 anddetermines an inverse tone mapping to be applied. The inverse tonemapping module 520 applies the inverse tone mapping to the luminancecomponent Y′_SDR 515 to obtain TL′_SDR 525.

The SL output module 530 receives the TL′_SDR 525 from the inverse tonemapping module 520, and applies the luminance component Y′_SDR 515 tothe TL′_SDR 525 to generate the SL signal 485.

FIG. 6A is a flow chart illustrating a process of encoding high dynamicrange image data into a single stream according to one embodiment. Thesteps shown in FIG. 6A can be performed by the image encoding system 130(e.g., image encoding system 130B of FIG. 2B). In other embodiments,some or all of the steps may be performed by other entities. Inaddition, some embodiments may perform the steps in parallel, performthe steps in different orders, or perform different steps.

The image encoding system 130 receives 610 input image data 115. In oneembodiment, the input image data 115 is represented in the RGB colorspace.

The image encoding system 130 performs 620 color conversion to obtainthe input image data 218 represented in the YUV color space.

The image encoding system 130 obtains 630 a SL signal 225 based on theinput image data 218 represented in the YUV color space. In oneembodiment, the image encoding system 130B implements data adaptivetuning to obtain the SL signal 225.

The image encoding system 130 obtains 640 a base signal 238 representedin the YUV color space based on the SL signal 225. In one embodiment,the base signal 238 is obtained by dividing the input image data 218represented in the YUV color space with the SL signal 225. Specifically,each of the luminance component and chroma components of the base signal238 can be divided by the SL signal 225.

The image encoding system 130 applies 660 data adaptive tuning to thebase signal 238 to obtain the regraded base signal 248. The dataadaptive tuning can be performed based on, for example, statistics ofthe base signal (e.g., luminance distribution). In one embodiment, thedata adaptive tuning is performed in the YUV color space.

The image encoding system 130 downsamples 670 the regraded base signal248 to obtain the downsampled base signal 265. In one embodiment, theimage encoding system 130 downsamples the regraded base signal 248represented in the 4:4:4 format into the 4:2:0.

The image encoding system 130 encodes 680 the downsampled base signal265. In one embodiment, the image encoding system 130 generates metadataor SEI describing parameters used for performing data adaptive tuning.The image encoding system 130 generates a data stream 140 by adding themetadata or the SEI to the encoded base signal. In another embodiment,the image encoding system 130 encodes the SL signal and adds the encodedSL signal to the data stream 140.

The image encoding system 130 transmits 690 the data stream 140including the encoded base signal and the SEI to the image decodingsystem 150.

FIG. 6B is a flow chart illustrating a process of decoding a singlestream into high dynamic range image data according to one embodiment.The steps shown in FIG. 6B can be performed by the image decoding system150 (e.g., image encoding system 150B of FIG. 4B). In other embodiments,some or all of the steps may be performed by other entities. Inaddition, some embodiments may perform the steps in parallel, performthe steps in different orders, or perform different steps.

The image decoding system 150 receives 613 a single data streamingincluding an encoded base signal. The image decoding system 150 decodes627 the encoded base signal to obtain a regraded downsampled base signal425 represented in the YUV color space.

The image decoding system 150 decodes 623 the encoded base signal toobtain SEI describing parameters used for performing DAT at the encodingsystem 130.

The image decoding system upsamples 637 the regraded downsampled basesignal 425 represented in the YUV color space to obtain a regraded basesignal 435 represented in the YUV color space. In one embodiment, theimage decoding system upsamples the regraded downsampled base signal 425in the 4:2:0 format into the 4:4:4 format.

The image decoding system 150 applies 647 inverse DAT to the regradedbase signal 435 to obtain the base signal 458 represented in the YUVcolor space. In one embodiment, the image decoding system 150 candetermine inverse DAT to the DAT applied in the image encoding system130 based on the SEI.

The image decoding system 150 generates 633 (or infers) the SL signal485 based on the SEI. In one embodiment, the image decoding system 150determines an inverse tone mapping to be applied based on the SEI. Theimage decoding system 150 applies the inverse tone mapping to aluminance component Y′_SDR 515 to obtain TL′_SDR 525, and then generatesthe SL signal 485 by applying the luminance component Y′_SDR 515 to theTL′_SDR 525.

The image decoding system 150 obtains 653 output image data 468represented in the YUV color space based on the SL signal 485 and thebase signal 458. In one embodiment, the output image data 468represented in the YUV color space is obtained by multiplying the basesignal 458 represented in the YUV color space with the SL signal 485.Specifically, each of the luminance component and chroma components ofthe base signal 458 can be multiplied by the SL signal 225.

The image decoding system 150 performs 663 color conversion on theoutput image data 468 represented in the YUV color space into the RGBcolor space.

FIG. 7A is a flow chart illustrating a process of encoding high dynamicrange image data into dual streams 140A, 140B, according to oneembodiment. The steps shown in FIG. 7A can be performed by the imageencoding system 130 (e.g., image encoding system 130C of FIG. 2C). Thesteps 610, 620, 630, 640, 660, and 670 are identical to the ones shownin FIG. 6A, except that the steps 675, 680, 690 of FIG. 6A are replacedwith steps 755, 780, 790, 795. Therefore, the detailed description ofthe steps 610, 620, 630, 640, 660, and 670 is omitted herein for thesake of brevity. In other embodiments, some or all of the steps may beperformed by other entities. In addition, some embodiments may performthe steps in parallel, perform the steps in different orders, or performdifferent steps.

In the embodiment shown in FIG. 7A, the image encoding system 130encodes 780 the downsampled base signal to generate a first data stream140A, irrespective of the SL signal. In addition, the image encodingsystem 150 encodes 755 the SL signal 225 to generate a second datastream 140B.

The image encoding system 130 transmits 790 the first data stream 140Aincluding the encoded base signal, and transmits 795 the second datastream 140B including the encoded SL signal to the image decoding system150.

FIG. 7B is a flow chart illustrating a process of decoding dual streamsinto high dynamic range image data according to one embodiment. Thesteps shown in FIG. 7B can be performed by the image decoding system 150(e.g., image encoding system 150C of FIG. 4C). The steps 627, 637, 647,653, and 663 in FIG. 7B are identical to the ones shown in FIG. 6B,except that the steps 613, 623, 633 of FIG. 6B are replaced with steps713, 717, 733. Therefore, the detailed description thereof is omittedherein for the sake of brevity. In other embodiments, some or all of thesteps may be performed by other entities. In addition, some embodimentsmay perform the steps in parallel, perform the steps in differentorders, or perform different steps.

In the embodiment shown in FIG. 7B, the image decoding system 150receives 613 a first data stream 140A including an encoded base signal,and receives 713 a second data stream 140B including encoded SL signal.The image decoding system 150 can decode 627 the encoded base signal inthe first data stream 140A. Similarly, the image decoding system 150 candecode 733 the encoded SL signal in the second data stream 140B togenerate the SL signal 485.

FIG. 8A is an example of a reconstructed sample image using aconventional high dynamic range encoding and decoding. FIG. 8B is anexample of a reconstructed sample image shown in FIG. 8A using enhancedhigh dynamic range encoding and decoding according to one embodiment. Asshown in FIG. 8B, a high dynamic range image can be encoded and decodedthrough standard codec (HEVC) according to one or more embodimentsdisclosed herein, while preserving details of the image (e.g., a line840).

Certain embodiments are described herein as including logic or a numberof components, modules (herein may be also referred to as “tools”), ormechanisms, for example, as illustrated in FIGS. 1-7. Modules (orcomponents) may constitute either software modules (e.g., code embodiedon a machine-readable medium or in a transmission signal) or hardwaremodules. A hardware module is tangible unit capable of performingcertain operations and may be configured or arranged in a certainmanner. In example embodiments, one or more computer systems (e.g., astandalone, client or server computer system) or one or more hardwaremodules of a computer system (e.g., a processor or a group ofprocessors) may be configured by software (e.g., an application orapplication portion) as a hardware module that operates to performcertain operations as described herein.

In some embodiments, a software module is implemented with a computerprogram product comprising a computer-readable medium containingcomputer program code, which can be executed by a computer processor forperforming any or all of the steps, operations, or processes described.

In some embodiments, a hardware module may be implementedelectronically. For example, a hardware module may comprise dedicatedcircuitry or logic that is permanently configured (e.g., as aspecial-purpose processor, such as a field programmable gate array(FPGA) or an application-specific integrated circuit (ASIC)) to performcertain operations. A hardware module may also comprise programmablelogic or circuitry (e.g., as encompassed within a general-purposeprocessor or other programmable processor) that is temporarilyconfigured by software to perform certain operations. Hardware moduleimplemented herein may be implemented in dedicated and permanentlyconfigured circuitry, or in temporarily configured circuitry (e.g.,configured by software).

Embodiments of the invention may also relate to an apparatus forperforming the operations herein. This apparatus may be speciallyconstructed for the required purposes, and/or it may comprise ageneral-purpose computing device selectively activated or reconfiguredby a computer program stored in the computer. Such a computer programmay be stored in a non transitory, tangible computer readable storagemedium, or any type of media suitable for storing electronicinstructions, which may be coupled to a computer system bus.Furthermore, any computing systems referred to in the specification mayinclude a single processor or may be architectures employing multipleprocessor designs for increased computing capability.

The above description is included to illustrate the operation of thepreferred embodiments and is not meant to limit the scope of theinvention. The scope of the invention is to be limited only by thefollowing claims. From the above discussion, many variations will beapparent to one skilled in the relevant art that would yet beencompassed by the spirit and scope of the invention.

What is claimed is:
 1. A method of regrading a high dynamic range videoto a lower dynamic range video using a data adaptive tuning process,comprising: converting, by an encoding system, a received video signalcomprising video frames of high-dynamic range image data to aluminance-chrominance (YUV) color space, wherein a luminance component Yof the high-dynamic range image data is denoted as a first signal;determining, for a selected video frame of the video signal, a set ofstatistical parameters of the video signal, drawn from frame-levelstatistics on a dynamic range of luminosity and colors of the video, inRGB and YUV components, which measure an extent to which the dynamicrange in both luminosity and in color space of the high dynamic rangevideo frame exceeds a lower dynamic range; generating a smoothedluminance signal from the first signal by performing a series ofoperations on the first signal, by: transforming, by the encodingsystem, the first signal using a selected data-adaptive opto-electricaltransfer function (OETF) to generate an OETF luminance signal, whereinthe selected data-adaptive OETF is selected based in part upon at leasta portion of the determined statistical parameters of the selected videoframe; performing, by the encoding system, a tone mapping on thegenerated OETF luminance signal to generate a tone mapped luminancesignal, the tone mapping based on a distribution characteristic of theOETF luminance signal; and transforming, by the encoding system, thetone mapped luminance signal using a selected data-adaptiveelectro-optical transfer function (EOTF) to generate the smoothedluminance signal, wherein the selected data-adaptive EOTF is an inverseof the selected data-adaptive OETF; modifying, by the encoding system,the received video signal using the smoothed luminance signal generatedfrom the first signal to generate a base signal, by dividing thereceived video signal by the smoothed luminance signal; generating, bythe encoding system, from the base signal, a regraded base signal byperforming a data adaptive tuning to the base signal, said data adaptivetuning of the base signal comprising applying an adaptive monotonicmapping of each component of the base signal for the selected frame,wherein a value of at least one coefficient of said adaptive monotonicmapping function is determined as a function of the statisticalparameters of the selected video frame; encoding, by the encodingsystem, the regraded base signal to generate an encoded base signal; andgenerating, by an encoding system, a first stream including the encodedbase signal, and metadata describing at least a portion of thestatistical parameters derived from the RGB and YUV components of thevideo and used in various tone maps and data adaptive tunings.
 2. Themethod of claim 1, wherein the various data adaptive tunings forregrading the high dynamic range video signal are measured and appliedat any combination of a block, slice, frame, GOP, scene, or sequencelevel.
 3. The method of claim 1, wherein generating the regraded basesignal by performing the data adaptive tuning to the base signalcomprises: determining a number of bins and binsizes to quantize theluminance signal into, the quantization based upon a luminositydistribution characteristic of the luminance signal; determining ahistogram of the luminance signal using the said number of bins andbinsizes; and determining a monotonic mapping of the base signal basedon features of the said histogram of the luminance signal with the saidnumber of bins and binsizes.
 4. The method of claim 1, wherein the setof statistical parameters comprises at least one of: a mean, covariance,peak brightness level, color range or gamut, histogram analysis of thevideo frame, and an aggregation of these statistics comprising at leastone of a min, mean, and max of these values over multiple framescorresponding to a GOP, scene, or video sequence.
 5. The method of claim1, wherein the regraded high dynamic range video signal is designed forpresentation on a target display, and wherein the added metadata derivedfrom statistical parameters include at least one of target peakbrightness and color range.
 6. The method of claim 1, wherein the dataadaptive transfer functions and added metadata correspond to onesdescribed in one of the following international standards: BT.709(high-definition TV), BT.2020 (ultra-high definition TV), ST-2084 (PQtone map), ST-2086 (static metadata), ST-2094 (dynamic metadata).
 7. Themethod of claim 1, wherein encoding the regraded high dynamic rangevideo signal is based on using a video coding standard selected fromthose by the ITU, ISO/IEC, Alliance for Open Media (AOM), Google VPX, orXiph Theora, and wherein the first stream further includes metadatadescribing statistical parameters for the data adaptive tuning that issufficient to recover image fidelity in the high dynamic range, themetadata added as supplemental enhancement information (SEI) messages inthe first stream.
 8. The method of claim 1, in which the tone mappingfunction applies a piecewise polynomial mapping to the OETF luminancesignal based upon the distribution characteristic.
 9. A method oftransmitting high dynamic range video data from a source device to asink device, the method comprising: regrading the high dynamic rangevideo to a lower dynamic range video, on a sequence, scene, GOP, orframe level, comprising: converting, by an encoding system, a receivedvideo signal comprising video frames of high-dynamic range image data toa luminance-chrominance (YUV) color space, wherein a luminance componentY of the high-dynamic range image data is denoted as a first signal;determining, for a selected video frame of the video signal, a set ofstatistical parameters of the video signal, drawn from frame-levelstatistics on a dynamic range of luminosity and colors of the video, inRGB and YUV components, which measure an extent to which the dynamicrange in both luminosity and in color space of the high dynamic rangevideo frame exceeds a lower dynamic range; generating a smoothedluminance signal from the first signal by performing a series ofoperations on the luminance signal, by: transforming, by the encodingsystem, the luminance signal using a selected data-adaptiveopto-electrical transfer function (OETF) to generate an OETF luminancesignal, wherein the selected data-adaptive OETF is selected based uponat least a portion of the determined set of statistical parameters ofthe selected video frame; performing, by the encoding system, a tonemapping on the generated OETF luminance signal to generate a tone mappedluminance signal, the tone mapping based on a distributioncharacteristic of the OETF luminance signal; and transforming, by theencoding system, the tone mapped luminance signal using a selecteddata-adaptive electro-optical transfer function (EOTF) to generate thesmoothed luminance signal, wherein the selected data-adaptive EOTF is aninverse of the data-adaptive OETF; modifying the high dynamic rangeimage data using the smoothed luminance signal generated from the firstsignal to generate a base signal in the red-green-blue (RGB) colorspace, by dividing each color component of the high-dynamic range imagedata represented in the red-green-blue (RGB) color space by the smoothedluminance signal; generating, by the encoding system, from the basesignal, a regraded base signal in the RGB color space by performing adata adaptive tuning to the base signal, said data adaptive tuning ofthe base signal comprising applying an adaptive monotonic mapping ofeach component of the base signal for the selected frame, wherein avalue of at least one coefficient of said adaptive monotonic mappingfunction to be applied on the selected frame is determined as a functionof the statistical parameters of the selected video frame, andcomprising: determining a number of bins and binsizes to quantize theluminance signal into, the quantization based upon a luminositydistribution characteristic of the luminance signal; determining ahistogram of the luminance signal using the said number of bins andbinsizes; and determining a monotonic mapping of the base signal basedon features of the said histogram of the luminance signal with the saidnumber of bins and binsizes; performing, by the encoding system, a RGBto luma-chroma (YUV) color transformation on the regraded base signal;encoding, by the encoding system, the regraded base signal to generate afirst stream; and transmitting, by the encoding system, the first streamto the sink device.
 10. The method of claim 9, wherein performing thedata adaptive tunings of the HDR video is based on the video data at anyof a sequence, scene, GOP, frame, slice, or block-level, and which maybe further adaptive to the capabilities of the sink device such as adisplay peak brightness, and color range.
 11. The method of claim 9,further comprising: generating metadata indicating at least a portion ofthe statistical parameters derived from the RGB and YUV components ofthe video and used for generating the smoothed luminance signal, whereinthe first stream is generated by encoding the regraded base signal andadding the metadata to the encoded base signal in supplementalenhancement information messages.
 12. The method of claim 9, furthercomprising: encoding the smoothed luminance signal to generate a secondstream different from the first stream; and transmitting the secondstream to the sink device.
 13. The method of claim 11, wherein thestatistical parameters include one of an input peak brightness, a targetpeak brightness, a number of bins, and bin sizes of histograms, that areadjusted according to the luminance component.
 14. The method of claim9, wherein the data adaptive transfer functions and added metadatacorrespond to ones described in one of the following internationalstandards: BT.709 (high-definition TV), BT.2020 (ultra-high definitionTV), ST-2084 (PQ tone map), ST-2086 (static metadata), ST-2094 (dynamicmetadata).
 15. The method of claim 9, wherein encoding the regraded highdynamic range video signal is based on using a video coding standardselected from those by the ITU, ISO/IEC, Alliance for Open Media (AOM),Google VPX, or Xiph Theora, and wherein the first stream furtherincludes metadata describing statistical parameters for the dataadaptive tuning that is sufficient to recover image fidelity in the highdynamic range, the metadata added as supplemental enhancementinformation (SEI) messages in the first stream.
 16. A computer readablenon-transitory storage medium storing instructions for processing asource high dynamic range video to a target dynamic range video,comprising: converting, by an encoding system, a received source videosignal comprising video frames of high-dynamic range image data to aluminance-chrominance (YUV) color space, wherein a luminance component Yof the high-dynamic range image data is denoted as a first signal;determining, for a selected video frame of the video signal, a set ofstatistical parameters of the video signal, drawn from frame-levelstatistics on the dynamic range of both the luminosity and colors of thevideo, in RGB and YUV components, indicative of the difference indynamic range between the source and target videos, in luminosity andcolor; generating a smoothed luminance signal from the first signal byperforming a series of operations on the first signal, by: transforming,by the encoding system, the luminance signal using a selecteddata-adaptive opto-electrical transfer function (OETF) to generate anOETF luminance signal, wherein the selected data-adaptive OETF isselected based upon at least a portion of the determined set ofstatistical parameters of the selected video frame; performing, by theencoding system, a tone mapping on the generated OETF luminance signalto generate a tone mapped luminance signal, the tone mapping based on adistribution characteristic of the OETF luminance signal; andtransforming, by the encoding system, the tone mapped luminance signalusing a selected data-adaptive electro-optical transfer function (EOTF)to generate the smoothed luminance signal, wherein the selecteddata-adaptive EOTF is an inverse of the data-adaptive OETF; modifying,by the encoding system, the high dynamic range image data using thesmoothed luminance signal generated from the first signal to generate abase signal by dividing the high-dynamic range image data represented bythe smoothed luminance signal; generating, by the encoding system, aregraded base signal by performing a data adaptive tuning to the basesignal, said data adaptive tuning of the base signal comprising applyingan adaptive monotonic mapping of each component of the base signal forthe selected frame, wherein a value of at least one coefficient of saidadaptive monotonic mapping function to be applied on the selected frameis determined as a function of the measured statistical parameters ofthe selected video frame; encoding, by the encoding system, the regradedbase signal to generate an encoded base signal; and generating metadatarepresenting at least a portion of statistical parameters extracted fromthe source video in comparison to the target video; generating, by anencoding system, a first stream including the encoded base signal andthe generated metadata.
 17. The computer readable non-transitory storagemedium of claim 16, wherein the stored instructions are further for:encoding the smoothed luminance signal; and generating metadata based onthe determined set statistical parameters, including describing thesmoothed luminance signal, wherein the first stream is generated byencoding the regraded base signal and adding the metadata to the encodedbase signal.
 18. The computer readable non-transitory storage medium ofclaim 16, wherein the stored instructions are further for: encoding thesmoothed luminance signal to generate a second stream different from thefirst stream; and transmitting the second stream to the sink device. 19.The computer readable non-transitory storage medium of claim 16, whereinthe stored instructions are further for: generating metadata includingthe portion of the set of statistical parameters used for generating thesmoothed luminance signal, wherein the first stream is generated byencoding the regraded base signal and adding the metadata to the encodedbase signal.
 20. The method of claim 19, wherein encoding the regradedbase signal is based on a video coding standard selected from those bythe ITU, ISO/IEC, Alliance for Open Media, Google VPX, or Xiph Theora,and wherein the metadata is added as supplemental enhancementinformation (SEI) messages in the bitstream, and corresponds to at leastone of BT.709, BT.2020, ST-2084, ST-2086, ST-2094.
 21. A method ofdecoding a high dynamic range video coded bitstream, wherein the codedbitstream was created by the method of claim
 1. 22. A method ofreceiving and decoding a transmitted high dynamic range video bitstream,wherein the transmitted bitstream was created by the method of claim 9.23. A computer readable non-transitory storage medium storinginstructions for decoding a high dynamic range video coded bitstream,said bitstream having been created by the method of claim 16.